Efficient architecture for layered VDR coding

ABSTRACT

In layered Visual Dynamic range (VDR) coding, inter-layer prediction requires several color-format transformations between the input VDR and Standard Dynamic Range (SDR) signals. Coding and decoding architectures are presented wherein inter-layer prediction is performed in the SDR-based color format, thus reducing computational complexity in both the encoder and the decoder, without compromising coding efficiency or coding quality.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 61/486,703 filed 16 May 2011, which is hereby incorporated byreference in its entirety.

TECHNOLOGY

The present invention relates generally to images. More particularly, anembodiment of the present invention relates to efficient color-spacetransformations in layered coding of high dynamic range images.

BACKGROUND

As used herein, the term ‘dynamic range’ (DR) may relate to a capabilityof the human psychovisual system (HVS) to perceive a range of intensity(e.g., luminance, luma) in an image, e.g., from darkest darks tobrightest brights. In this sense, DR relates to a ‘scene-referred’intensity. DR may also relate to the ability of a display device toadequately or approximately render an intensity range of a particularbreadth. In this sense, DR relates to a ‘display-referred’ intensity.Unless a particular sense is explicitly specified to have particularsignificance at any point in the description herein, it should beinferred that the term may be used in either sense, e.g.interchangeably.

As used herein, the term high dynamic range (HDR) relates to a DRbreadth that spans the some 14-15 orders of magnitude of the humanvisual system (HVS). For example, well adapted humans with essentiallynormal (e.g., in one or more of a statistical, biometric orophthalmological sense) have an intensity range that spans about 15orders of magnitude. Adapted humans may perceive dim light sources of asfew as a mere handful of photons. Yet, these same humans may perceivethe near painfully brilliant intensity of the noonday sun in desert, seaor snow (or even glance into the sun, however briefly to preventdamage). This span though is available to ‘adapted’ humans, e.g., thosewhose HVS has a time period in which to reset and adjust.

In contrast, the DR over which a human may simultaneously perceive anextensive breadth in intensity range may be somewhat truncated, inrelation to HDR. As used herein, the terms ‘visual dynamic range’ or‘variable dynamic range’ (VDR) may individually or interchangeablyrelate to the DR that is simultaneously perceivable by a HVS. As usedherein, VDR may relate to a DR that spans 5-6 orders of magnitude. Thuswhile perhaps somewhat narrower in relation to true scene referred HDR,VDR nonetheless represents a wide DR breadth. As used herein, the term‘simultaneous dynamic range’ may relate to VDR.

Until fairly recently, displays have had a significantly narrower DRthan HDR or VDR. Television (TV) and computer monitor apparatus that usetypical cathode ray tube (CRT), liquid crystal display (LCD) withconstant fluorescent white back lighting or plasma screen technology maybe constrained in their DR rendering capability to approximately threeorders of magnitude. Such conventional displays thus typify a lowdynamic range (LDR), also referred to as a standard dynamic range (SDR),in relation to VDR and HDR.

Advances in their underlying technology however allow more moderndisplay designs to render image and video content with significantimprovements in various quality characteristics over the same content,as rendered on less modern displays. For example, more modern displaydevices may be capable of rendering high definition (HD) content and/orcontent that may be scaled according to various display capabilitiessuch as an image scaler. Moreover, some more modern displays are capableof rendering content with a DR that is higher than the SDR ofconventional displays.

For example, some modern LCD displays have a backlight unit (BLU) thatcomprises a light emitting diode (LED) array. The LEDs of the BLU arraymay be modulated separately from modulation of the polarization statesof the active LCD elements. This dual modulation approach is extensible(e.g., to N-modulation layers wherein N comprises an integer greaterthan two), such as with controllable intervening layers between the BLUarray and the LCD screen elements. Their LED array based BLUs and dual(or N-) modulation effectively increases the display referred DR of LCDmonitors that have such features.

Such “HDR displays” as they are often called (although actually, theircapabilities may more closely approximate the range of VDR) and the DRextension of which they are capable, in relation to conventional SDRdisplays represent a significant advance in the ability to displayimages, video content and other visual information. The color gamut thatsuch an HDR display may render may also significantly exceed the colorgamut of more conventional displays, even to the point of capablyrendering a wide color gamut (WCG). Scene related HDR or VDR and WCGimage content, such as may be generated by “next generation” movie andTV cameras, may now be more faithfully and effectively displayed withthe “HDR” displays (hereinafter referred to as ‘HDR displays’).

As with the scalable video coding and HDTV technologies, extending imageDR typically involves a bifurcate approach. For example, scene referredHDR content that is captured with a modern HDR capable camera may beused to generate an SDR version of the content, which may be displayedon conventional SDR displays. In one approach, generating the SDRversion from the captured VDR version may involve applying a tonemapping operator (TMO) to intensity (e.g., luminance, luma) relatedpixel values in the HDR content. In a second approach, as described inU.S. provisional application 61/376,907 “Extending Image Dynamic Range”,by W. Gish et al., herein incorporated by reference for all purposes,generating an SDR image may involve applying an invertible operator (orpredictor) on the VDR data. To conserve bandwidth or for otherconsiderations, transmission of the actual captured VDR content may notbe a best approach.

Thus, an inverse tone mapping operator (iTMO), inverted in relation tothe original TMO, or an inverse operator in relation to the originalpredictor, may be applied to the SDR content version that was generated,which allows a version of the VDR content to be predicted. The predictedVDR content version may be compared to originally captured HDR content.For example, subtracting the predicted VDR version from the original VDRversion may generate a residual image. An encoder may send the generatedSDR content as a base layer (BL), and package the generated SDR contentversion, any residual image, and the iTMO or other predictors as anenhancement layer (EL) or as metadata.

Sending the EL and metadata, with its SDR content, residual andpredictors, in a bitstream typically consumes less bandwidth than wouldbe consumed in sending both the HDR and SDR contents directly into thebitstream. Compatible decoders that receive the bitstream sent by theencoder may decode and render the SDR on conventional displays.Compatible decoders however may also use the residual image, the iTMOpredictors, or the metadata to compute a predicted version of the HDRcontent therefrom, for use on more capable displays

In such layered VDR coding, signals may be represented at different bitdepths, at different color spaces, and at different chroma sub samplingformats, all of which may force a variety of computer-intensivetransformations from a first color format to a second color format.

As used herein, the term “color format” relates to a colorrepresentation that comprises two variables: a) a color space variable(for example: RGB, YUV, YCbCr, and the like) and a chroma subsamplingvariable (for example: 4:4:4, 4:2:0, and the like.) For example, a VDRsignal may have an RGB 4:4:4 color format, while an SDR signal may havea YCbCr 4:2:0 color format. Embodiments related to methods andarchitectures for efficient color-format processing in VDR layeredcoding are presented herein.

In an example embodiment, in the encoder, both the SDR-to-VDR predictorand a residual non-linear equalizer operate in the SDR color format.This allows the decoder to require far less color-format relatedcomputations, with no degradation in video quality.

The approaches described in this section are approaches that could bepursued, but not necessarily approaches that have been previouslyconceived or pursued. Therefore, unless otherwise indicated, it shouldnot be assumed that any of the approaches described in this sectionqualify as prior art merely by virtue of their inclusion in thissection. Similarly, issues identified with respect to one or moreapproaches should not assume to have been recognized in any prior art onthe basis of this section, unless otherwise indicated.

BRIEF DESCRIPTION OF THE DRAWINGS

An embodiment of the present invention is illustrated by way of example,and not in way by limitation, in the figures of the accompanyingdrawings and in which like reference numerals refer to similar elementsand in which:

FIG. 1 depicts an example data flow for a VDR-SDR system, according toan embodiment of the present invention;

FIG. 2 depicts an example layered VDR encoding system according to anembodiment of the present invention;

FIG. 3 depicts an example layered VDR decoding systems according to anembodiment of the present invention;

FIG. 4A depicts an example of color-format processing in a layered VDRencoding system;

FIG. 4B depicts an example of color-format processing in a layered VDRdecoding system;

FIG. 5A depicts an example of color-format processing in a layered VDRencoding system according to one embodiment of this invention;

FIG. 5B depicts an example of color-format processing in a layered VDRdecoding system according to one embodiment of this invention.

DESCRIPTION OF EXAMPLE EMBODIMENTS

Given a pair of corresponding VDR and SDR images, such as images thatrepresent the same scene, each at different levels of dynamic range,improved coding of the residual signal in layered VDR coding isachieved. The VDR image is coded by combining a base layer (e.g., theSDR image) and a residual as an enhancement layer. In an embodiment, theenhancement layer comprises a difference between the original VDR imageand a version thereof that is predicted, e.g., from the base layer. Inthe following description, for the purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the present invention. It will be apparent, however,that the present invention may be practiced without these specificdetails. In other instances, well-known structures and devices are notdescribed in exhaustive detail, in order to avoid unnecessarilyoccluding, obscuring, or obfuscating the present invention.

Overview

Example embodiments described herein relate to the layered coding ofimages with high dynamic range. An embodiment applies layer predictionand the non-linear quantization of the residual in the color format ofthe SDR signal, thus reducing computation complexity related tocolor-format transformations without compromising picture quality orcoding efficiency.

Example VDR-SDR System

FIG. 1 depicts an example data flow in a VDR-SDR system 100, accordingto an embodiment of the present invention. An HDR image or videosequence is captured using HDR camera 110. Following capture, thecaptured image or video is processed by a mastering process to create atarget VDR image 125. The mastering process may incorporate a variety ofprocessing steps, such as: editing, primary and secondary colorcorrection, color transformation, and noise filtering. The VDR output125 of this process represents the director's intend on how the capturedimage will be displayed on a target VDR display.

The mastering process may also output a corresponding SDR image 145,representing the director's intend on how the captured image will bedisplayed on a legacy SDR display. The SDR output 145 may be provideddirectly from mastering circuit 120 or it may be generated by a separateVDR-to-SDR converter 140.

In an example embodiment, the VDR 125 and SDR 145 signals are input intoan encoder 130. Encoder 130 creates a coded bitstream which reduces thebandwidth required to transmit the VDR and SDR signals. Moreover,encoder 130 functions to encode a signal that allows a correspondingdecoder 150 to decode and render either the SDR or VDR signalcomponents. In an example implementation, encoder 130 may be a layeredencoder, such as one of those defined by the MPEG-2 and H.264 codingstandards, which represents its output as a base layer, an optionalenhancement layer, and metadata. As defined herein, the term “metadata”relates to any auxiliary information that is transmitted as part of thecoded bitstream and assists a decoder to render a decoded image. Suchmetadata may include, but are not limited to, such data as: color spaceor gamut information, dynamic range information, tone mappinginformation, or other predictor and quantizer operators, such as thosedescribed herein.

On the receiver, decoder 150 uses the received coded bitstreams andmetadata to render either an SDR image 157 or a VDR image 156, accordingto the capabilities of the target display. For example, an SDR displaymay use only the base layer and the metadata to render an SDR image. Incontrast, a VDR display may use information from all input layers andthe metadata to render a VDR signal.

FIG. 2 depicts an example implementation of encoder 130 incorporatingthe methods of this invention. In FIG. 2, VDR input 205 may berepresented as 16-bit (fixed point or floating point) RGB 4:4:4 whileSDR input 207 is typically 8-bit YCbCr (or YUV), 4:2:0, ITU Rec. 709data; however the methods of this embodiment apply to other VDR and SDRrepresentations as well. For example, some implementations may use anenhanced SDR input, SDR′, which may have the same color space (primariesand white point) as SDR, but may use high precision, say 12-bits perpixel, with all color components at full spatial resolution (e.g., 4:4:4RGB). The SDR input 207 is applied to compression system 240. Dependingon the application, compression system 240 can be either lossy, such asaccording to the H.264 or MPEG-2 standards, or lossless. The output ofthe compression system 240 may be transmitted as a base layer 225. Toreduce drift between the encoded and decoded signals, encoder 130 mayfollow compression process 240 with a corresponding decompressionprocess 230. Signal 235 represents the SDR input as it will be receivedby a decoder. Predictor 250, as described for example in U.S.provisional application 61/475,359, “Multiple color channel multipleregression predictor”, by G-M. Su et al., using input VDR 205 and SDR235 data will create signal 257 which represents an approximation orestimate of input VDR 205. Adder 260 subtracts the predicted VDR 257from the original VDR 205 to form output residual signal 265. Residual265 may also be coded by another lossy or lossless encoder 220, such asthose defined by the MPEG standards, and may be multiplexed in theoutput bit stream and transmitted to the decoder as an enhancementlayer.

Predictor 250 may also provide the prediction parameters being used inthe prediction process as metadata 255. Since prediction parameters maychange during the encoding process, for example, on a frame by framebasis, or on a scene by scene basis, these metadata may be transmittedto the decoder as part of the data that also include the base layer andthe enhancement layer.

Residual 265 represents the difference between two VDR signals, thus itis expected to be represented by more than 8-bits per color component.In many possible implementations, encoder 220 may not be able to supportthe full dynamic range of this residual signal. In an exampleimplementation, the residual may be 16 bits and the residual encoder 220may be a standard H.264, 8-bit, encoder. In order for encoder 220 toaccommodate the dynamic range of residual 265, quantizer 210 quantizesresidual 265 from its original bit-depth representation (say 12 or 16bits) to a lower bit-depth representation. The quantizer parameters mayalso be multiplexed into the metadata bitstream 255.

In one possible implementation, one may pre-process residual 265 by anon-linear quantizer, such as the one described in U.S. provisionalapplication 61/478,836, “Non Linear VDR Residual Quantizer,” by G-M Suet al.

FIG. 3 depicts in more detail an example implementation of decoder 150.Decoding system 300 receives a coded bitstream 320 that may combine abase layer 337, an enhancement layer (or residual) 332, and metadata335, which are extracted following decompression 330. For example, in aVDR-SDR system, the base layer 337 may represent the SDR representationof the coded signal and the metadata 335 may include information relatedthe prediction (250) and quantization (210) steps used in the encoder.Residual 332 is decoded (340), de-quantized (350) as decodedde-quantized residual 365, and added (360) to the output 395 of thepredictor 390 to generate the output VDR signal 370. As in the encoder,VDR and SDR signals may be represented using different color formats,such as RGB 4:4:4 for the VDR signal and YCbCr (or YUV) 4:2:0 for theSDR signal.

Color-Format Transformations

As depicted in FIG. 2, SDR and VDR input signals to encoder 130 may havedifferent color format representations. The goal of encoder 130 is tonot only compress the signals as efficiently as possible, but alsopreserve image quality. Compression blocks 240 and 220 operate far moreefficiently when the input signals are represented in a Luma-Chromacolor space (such as YUV or YCbCr); however, predictor 250 and quantizer210 may operate in either the VDR or the SDR color spaces. However,regardless of what color format space is chosen, both the encoder andthe decoder will be required to perform color-format transformations. Itis one goal of this invention to reduce the computation requirements forsuch color-format transformations.

FIG. 4A depicts the color-format transformation operations when thepredictor 250 operates in the VDR color format. In FIG. 4A, in block410, SDR input 407 is transformed to match the VDR color format. Forexample, if VDR input 405 is in RGB 4:4:4 format and SDR input 407 is inYCbCr 4:2:0 format, then block 410 may convert SDR YCbCr 4:2:0 to SDRRGB 4:4:4 in two steps: a) chroma up-sample YCbCr 4:2:0 to YCbCr 4:4:4using any of the well-known up-sampling and interpolation techniques,and b) color transform YCbCr 4:4:4 to RGB 4:4:4 using well known colortransform operations.

From FIG. 4A, the output 425 of the predictor and the residual 435 willalso be in VDR color format (say, RGB 4:4:4). Residual encoder may be anMPEG (say MPEG-2, MPEG-4, or H.264) encoder operating in YCbCr 4:2:0format. Such an implementation requires that residual 435 beingtransformed to match the color format of the residual encoder. Block 440transforms input signal from the VDR input format (say, RGB 4:4:4) tothe color format of the residual encoder (say, YCbCr 4:2:0). Forexample, if the color format of the residual encoder is YCbCr 4:2:0,then block 440 may convert RGB 4:4:4 into YCbCr 4:2:0 in two steps: a)color transform RGB 4:4:4 to YCbCr 4:4:4 using well known colortransform operations, and b) chroma down-sample YCbCr 4:4:4 to YCbCr4:2:0.

Block 440 may be positioned before the quantizer 450 (as depicted) orafter the quantizer 450 (not shown). Because of the color sub-samplingin block 440, positioning color-format transform block 440 before thequantizer 450 (before residual encoder 460) reduces significantly thecomputations on the quantizer as well.

FIG. 4B depicts color format operations on a decoder corresponding tothe encoder depicted in FIG. 4A. Since the encoder predictor 420operated in VDR color format, on the decoder, predictor 485 (with output487) needs to operate in VDR color format as well. This configurationrequires two color format transformation operations: a) block 475, whichtransforms the base layer from an SDR color format to the VDR colorformat 477, and b) block 490, which transforms the de-quantized residual(from residual decoder 470 and de-quantizer 480) from the residualencoder's color format to the VDR color format.

From FIGS. 4A and 4B, this VDR coding implementation requires twocolor-format transformations in the encoder and two color-formattransformations in the decoder. A far more efficient configurationaccording to an example embodiment of our invention is depicted in FIGS.5A and 5B.

FIG. 5A depicts an example color-format transformation process for a VDRencoder when the predictor 250 (520) operates in the SDR color format515. In this implementation, there are no color-format transformationson the SDR input 507. Instead, transformation block 510 transforms thecolor format of the VDR input 505 to match the color format of the SDRinput. For example, if VDR input is in RGB 4:4:4 and the SDR input is inYCbCr 4:2:0, then block 510 will preserve the bit precision of the VDRsignal, but will transform it to YCbCr 4:2:0. In this implementation,both the output 525 of the predictor 520 and residual output 535 will bein YCbCr 4:2:0 format. If the residual encoder 560 (after quantizer 550)operates in the same color format of the SDR input (say, YCbCr 4:2:0),then no additional color format transformations are needed in theencoder.

FIG. 5B depicts color format operations on a decoder corresponding tothe encoder depicted in FIG. 5A. Coded residual signal 567 is decoded byresidual decoder 570 and then de-quantized by de-quantizer 580. Sincethe encoder predictor 520 operates in SDR color format, predictor 585needs to operate in SDR color format 577 as well. Thus the output signal597, representing an enhancement layer 567 added to the output 587 ofthe predictor 585, will also be in the SDR color format. Finally,transformation operations in block 590 transforms signal 597 from theSDR color format (say, YCbCr 4:2:0) to the desired color format foroutput VDR signal 598. The final output format can be the same as theinput VDR color format, say RGB 4:4:4, or a different one, for exampleone matching the requirements of a target VDR display.

In an example implementation, assuming that VDR input is in RGB 4:4:4color format and that the SDR input is in YCbCr 4:2:0 format, comparingthe computational requirements between the VDR color format-based codingmethod (400) and the SDR color-format based coding method (500) we canderive the following computational savings:

On the encoder: (a) Due to chroma subsampling, predictor 520 and adder530 operate on half the chroma pixel samples than predictor 420 andadder 430, and (b) encoder 400A requires an additional “Match VDR colorformat” block than encoder 500A.

On the decoder: (a) Due to chroma subsampling, predictor 585 and adder595 operate on half the chroma pixel samples than predictor 485 andadder 495 (which outputs VDR 497), and (b) decoder 400B requires anadditional “Match VDR color format” block than decoder 500B.Furthermore, in decoder 500B, color-format transformation block 590 canbe eliminated or combined by another color transformation block,depending on the application and the requirements of the devicereceiving the output VDR signal 598.

Example Computer System Implementation

Embodiments of the present invention may be implemented with a computersystem, systems configured in electronic circuitry and components, anintegrated circuit (IC) device such as a microcontroller, a fieldprogrammable gate array (FPGA), or another configurable or programmablelogic device (PLD), a discrete time or digital signal processor (DSP),an application specific IC (ASIC), and/or apparatus that includes one ormore of such systems, devices or components. The computer and/or IC mayperform, control or execute instructions relating to color-formattransformations, such as those described herein. The computer and/or ICmay compute, any of a variety of parameters or values that relate tocolor-format transformations as described herein. The image and videodynamic range extension embodiments may be implemented in hardware,software, firmware and various combinations thereof.

Certain implementations of the invention comprise computer processorswhich execute software instructions which cause the processors toperform a method of the invention. For example, one or more processorsin a display, an encoder, a set top box, a transcoder or the like mayimplement color format transformation methods as described above byexecuting software instructions in a program memory accessible to theprocessors. The invention may also be provided in the form of a programproduct. The program product may comprise any medium which carries a setof computer-readable signals comprising instructions which, whenexecuted by a data processor, cause the data processor to execute amethod of the invention. Program products according to the invention maybe in any of a wide variety of forms. The program product may comprise,for example, physical media such as magnetic data storage mediaincluding floppy diskettes, hard disk drives, optical data storage mediaincluding CD ROMs, DVDs, electronic data storage media including ROMs,flash RAM, or the like. The computer-readable signals on the programproduct may optionally be compressed or encrypted.

Where a component (e.g. a software module, processor, assembly, device,circuit, etc.) is referred to above, unless otherwise indicated,reference to that component (including a reference to a “means”) shouldbe interpreted as including as equivalents of that component anycomponent which performs the function of the described component (e.g.,that is functionally equivalent), including components which are notstructurally equivalent to the disclosed structure which performs thefunction in the illustrated example embodiments of the invention.

EQUIVALENTS, EXTENSIONS, ALTERNATIVES AND MISCELLANEOUS

Example embodiments that relate to applying color format transformationsin coding and decoding VDR and SDR images are thus described. In theforegoing specification, embodiments of the present invention have beendescribed with reference to numerous specific details that may vary fromimplementation to implementation. Thus, the sole and exclusive indicatorof what is the invention, and is intended by the applicants to be theinvention, is the set of claims that issue from this application, in thespecific form in which such claims issue, including any subsequentcorrection. Any definitions expressly set forth herein for termscontained in such claims shall govern the meaning of such terms as usedin the claims. Hence, no limitation, element, property, feature,advantage or attribute that is not expressly recited in a claim shouldlimit the scope of such claim in any way. The specification and drawingsare, accordingly, to be regarded in an illustrative rather than arestrictive sense.

The invention claimed is:
 1. In a layered video encoder, a method ofcoding video signals to reduce color format conversion operations, themethod comprising: receiving a first video signal having a first dynamicrange in a first color space and a first chrominance sampling format ofa first color format; receiving a second video signal corresponding tothe same content as the first video signal, wherein the second videosignal has a second dynamic range in a second color space and a secondchrominance sampling format of a second color format, wherein the seconddynamic range is smaller than the first dynamic range, wherein thesecond chrominance sampling format of the second color format isdifferent from the first chrominance sampling format of the first colorformat, and wherein the second color space of the second color format isdifferent from the first color space of the first color format;generating a third video signal based on the first video signal byconverting the first video signal in the first color space of the firstcolor format into the third video signal in the second color space andthe second chrominance sampling format of the second color format whilepreserving the first dynamic range; determining a predicted video signalof the third video signal in the second color space of the second colorformat based at least on the second video signal, the predicted videosignal having the first dynamic range in the second color space and thesecond chrominance sampling format of the second color format; codingthe first video signal using a coded base layer stream and a codedenhancement layer stream of a coded bitstream, wherein the coded baselayer stream comprises a coded representation of the second videosignal, and the coded enhancement layer stream comprises a codedrepresentation of a residual between the third video signal and thepredicted video signal; causing a recipient device of the codedbitstream to generate a decoded image from the coded bitstream forrendering.
 2. The method of claim 1, wherein the first video signalcomprises a visual dynamic range (VDR) signal and the second videosignal comprises a standard dynamic range (SDR) signal.
 3. The method ofclaim 2, wherein the VDR signal comprises an RGB 4:4:4 color format andthe SDR signal comprises a YUV or YCbCr 4:2:0 color format.
 4. Themethod of claim 1, wherein determining the predicted video signalcomprises using a multiple-color channel multiple-regression (MMR)predictor.
 5. The method of claim 1, wherein generating the codedenhancement layer stream further comprises: quantizing the residual witha non-linear quantizer to generate a quantized residual signal with thesecond dynamic range; and coding the quantized residual signal with avideo encoder to generate the coded enhancement layer stream.
 6. Themethod of claim 1, wherein in a decoder, a method of decoding the codedfirst video signal comprises: decoding the coded base layer stream togenerate a decoded base layer signal in the second color format;decoding the coded enhancement layer stream to generate a decodedenhancement layer signal in the second color format; determining, usinga predictor, a decoder predicted signal in the second color format inresponse to the decoded base layer signal; generating an intermediateoutput signal based on the decoded enhancement layer signal and thedecoder predicted signal, the intermediate output signal having thefirst dynamic range and the second color format; and determining adecoded representation of the first signal by transforming theintermediate output signal from the second color format into a finaloutput color format, the final output color format being different fromthe second color format.
 7. The method of claim 6, wherein the finaloutput color format comprises the first color format.
 8. A method ofdecoding a visual dynamic range (VDR) video signal in a video codeccomprising a base layer and an enhancement layer, the method comprising:receiving a coded VDR stream comprising a base layer stream and anenhancement layer stream, wherein both the base layer stream and theenhancement layer stream are to be used to generate a reconstructedversion of a first video signal having a first dynamic range in a firstcolor space and a first chrominance sampling format of a first colorformat; wherein the base layer stream comprises a coded representationof a second video signal having a second dynamic range in a second colorspace and a second chrominance sampling format of a second color format;wherein the enhancement layer stream comprises a coded representation ofa residual between a third video signal and a predicted video signal ofthe third video signal; wherein the third video signal was generated byan upstream encoder based on the first video signal by converting thefirst video signal in the first color space of the first color formatinto the third video signal in the second color space and the secondchrominance sampling format of the second color format while preservingthe first dynamic range; wherein the predicted video signal of the thirdvideo signal was determined by the upstream encoder based at least onthe second video signal, the predicted video signal having the firstdynamic range in the second color space and the second chrominancesampling format of the second color format; wherein the secondchrominance sampling format of the second color format is different fromthe first chrominance sampling format of the first color format andwherein the second color space of the second color format is differentfrom the first color space of the first color format; decoding the baselayer stream to generate the second video signal having the seconddynamic range in the second color space and the second chrominancesampling format of the second color format; decoding the enhancementlayer stream to generate the residual between the third video signal andthe predicted video signal of the third video signal; generating, usinga predictor, the predicted video signal of the third video signal inresponse to the decoded second video signal; generating a VDRintermediate output signal in the second color space and the secondchrominance sampling format of the second color format based on theresidual between the third video signal and the predicted video signalof the third video signal and the predicted video signal of the thirdvideo signal; transforming the VDR intermediate signal from the secondcolor space and the second chrominance sampling format of the secondcolor format to the first color space and the first chrominance samplingformat of the first color format to generate the reconstructed versionof the first video signal having the first dynamic range in the firstcolor space and the first chrominance sampling format of the first colorformat; causing a reconstructed image in the reconstructed version ofthe first video signal to be rendered.
 9. The method of claim 8, whereinthe second color format comprises a YUV or YCbCr 4:2:0 color format. 10.The method of claim 8, wherein the first color format comprises an RGB4:4:4 color format.
 11. The method of claim 8, wherein generating thedecoded VDR enhancement layer signal further comprises: decoding theenhancement layer stream with a video decoder to generate an SDRenhancement layer signal; dequantizing the decoded SDR enhancement layersignal with a non-linear dequantizer to generate the decoded VDRenhancement layer signal.
 12. An apparatus comprising one or moreprocessors and one or more non-volatile computer-readable storage mediastoring instructions, which when executed by the one or more processors,cause the one or more processors to perform a method as recited inclaim
 1. 13. An apparatus comprising one or more processors and one ormore non-volatile computer-readable storage media having storedinstructions, which when executed by the one or more processors, causethe one or more processors to perform a method as recited in claim 8.14. One or more non-volatile computer-readable storage media havingstored instructions, which when executed by one or more processors,cause the one or more processors to perform a method as recited inclaim
 1. 15. One or more non-volatile computer-readable storage mediahaving stored instructions, which when executed by one or moreprocessors, cause the one or more processors to perform a method asrecited in claim 8.